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area image. 
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METHOD AND APPARATUS 
FOR DUAL RESOLUTION FO VEATING 
DISPLAY OUTPUT 



CROSS REFERENCE TO RELATED APPLICATIONS 

This application relates to Provisional Application Serial Number 60/120,268, entitled, 
"CMOS APS FO VEATING ARRAY WITH RANDOM X- Y FOVEAL AREA" filed on 
February 24, 1999, which is assigned to the same assignee and is incorporated by reference 
5 herein. Applicant claims the benefit of the priority filing date of February 24, 1999 pursuant to 
35 U.S.C. § 119(e)(1). 



FIELD OF THE INVENTION 

The present invention is related to the field of solid state imagers. More specifically, the 
10 present invention is related to providing concurrent dual resolution outputs of imaging devices. 

BACKGROUND OF THE INVENTION 

Traditionally, an Active Pixel Sensor (APS) imager is formed by arranging photoelectric 
or photosensitive, solid state devices in a rectangular or square array. Each photoelectric solid 
15 state device, termed a "pixel," converts photonic energy into an electrical charge and outputs the 



electric charge when the pixel is selected. Imager arrays of 512 x 512, 1024 x 1024 and 2048 x 
2048 pixels are well known in the art. 
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Typically, an image is created by serially reading out the electric charge of each pixel cell 
in the imager array by selecting a row and then sequentially reading the electrical charge of each 
pixel cell in the selected row. After the last pixel cell in the selected row is read, a next row is 
selected and the pixel cell read out process repeats, row by row, until each pixel cell has been 
5 read. In one method of sequentially reading the image array rows, entitled "progressive" 

scanning, after each row is completely scanned, the pixel cells in the next adjacent row are read 
or scanned. Thus, the array rows are scanned in sequential order, i.e., row 1, 2, 3 through M. In 
another scanning method, entitled "interlaced" scanning, after a selected row is scanned, the scan 
proceeds to a next non-contiguous row. In one example of interlaced scanning, alternate rows 
10 are scanned in the order 1, 3, 5, through M-l and then 2, 4, 6, through M, where M is an even 
number of rows. 

To track small targets with good resolution over the imaging area, a high-resolution 
imager containing a large number of pixel cells is necessary. For imager arrays of the same 
overall dimension, the number of pixel cells contained in the imager array determines the 

15 resolution of the imager. For example, an array of 2048 x2048 pixels has a resolution sixteen 
times greater than an array of 512 x 512. High-resolution imagery is typically desired to obtain a 
clear and sharp image. As the number of pixel cells increases however, the time necessary to 
read the entire image significantly increases. This time to read the image, known as the image 
Frame time, is determined by the pixel read out speed and the total number of pixels in the 

20 imager array. As the pixel read out speed is typically fixed by the hardware used, the Frame time 
is determined by the number of pixel cells that must be read. Accordingly, as the resolution 
increases, the Frame time also increases. Increased Frame time causes moving images to appear 
to change in a sporadic and irregular manner. For example, objects moving faster that the Frame 
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time may appear to jump from one position to another rather than smoothly transitioning 
between these positions. Hence, there is a need to provide a high-resolution imager with a Frame 
time of a short enough duration to capture transitions smoothly while maintaining an acceptable 
image quality field of view. 

5 

SUMMARY OF THE INVENTION 

A method and apparatus for providing dual-resolution read out from a solid state CMOS 
Active Pixel Sensor (APS) imager is presented. The present invention provides for a first output 
that generates a low resolution, wide field of view (WFOV) image, and a second output that 

10 generates a higher resolution, narrow field of view (NFOV) image. The high-resolution output is 
generated using the outputs of individual pixel cells in the image array, while the low-resolution 
output is generated by combining a plurality of pixel cells into a single output. Using a 
WFOV and an NFOV image read out processor, an entire image may be quickly scanned and 
displayed using a low resolution image, while at least one designated area of the imager array 

15 may be displayed using a high resolution image 

In one embodiment of the invention, the high-resolution output may overwrite or replace 
a low-resolution image. In a second embodiment of the invention, the high-resolution image 
may be directed to a second device to display high-resolution images independently of a lower 
resolution image. 

20 



BRIEF DESCRIPTION OF THE DRAWINGS 
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The advantages, nature and various additional features of the invention will appear more 
fully upon consideration of the illustrative embodiments to be described in detail in connection 
with the accompanying drawings. In the drawings: 

Figure 1 illustrates a prior art pixel cell and associated control logic used in APS pixel 

5 arrays; 

Figure 2 illustrates a block diagram implementation of a dual-resolution foveating array 
in accordance with the principles of the invention; 

Figure 3a illustrates an exemplary embodiment of dual-resolution circuitry according to 
the principles of the invention; 
10 Figure 3b illustrates a second exemplary embodiment of the dual-resolution circuitry 

according to the principles of the invention; 

Figure 4 illustrates an exemplary embodiment of the dual-resolution circuit logic in a 
foveating array in accordance with the principles of the invention; 

Figure 5a depicts a scene to be digitally captured; 
15 Figure 5b depicts a low-resolution digital image of the scene illustrated in Figure 5a; 

Figure 5c illustrates an exemplary high-resolution region within the low-resolution image 
illustrated in Figure 5b; 

Figure 5d illustrates a high-resolution image of the high-resolution region of the scene 
illustrated in Figure 5a; 

20 Figure 5e illustrates a foveating image created in accordance with an exemplary operation 

of the dual-resolution array in accordance with the principles of the invention; and 

Figure 6 illustrates an exemplary system using the dual-resolution foveating array in 
accordance with the principles of the invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

A CMOS APS imager includes a pixel area and a read out/processing logic. The pixel 
5 area consists of a photosensitive area for photoelectron generation in a matrix configuration, 
addressing circuitry and amplification circuits that operate to address and access individual pixel 
cells. The read out and processing area provides for signal conditioning of input and output data. 
Figure 1 illustrates a typical CMOS APS pixel cell and associated processing circuitry and logic. 
As illustrated, solid state photoelectric device 100 is used to collect photonic energy over a 

10 designated time period and convert the collected energy into an electrical charge. The electrical 
charge is then amplified by amplifier circuit 106, and by the appropriate selection of Row Select 
Switch 104 and Column Select Switch 102, the amplified electrical charge is transferred to video 
output line 110. After the read out of photocell 100 occurs, reset switch 108 is activated to return 
the pixel cell charge to a nominal value. This reset prevents electrical charge collected before 

15 the read out from influencing electrical charge collected during the next period of photon 
collection. 

Figure 2 illustrates a typical row and column arrangement of pixel cells 100 and 
corresponding logic circuitry used to sequentially read out the electrical data from the pixel cells 
100. In this embodiment, a plurality of pixel cells 100 are arranged in an M x N array of twelve 
20 pixel cells 100 per row in eight rows --/.e., 12x8 imager array. For clarity only a sample of the 
pixel cells 100 within the imager array are specifically labeled. Thus, pixel cells 240, 241, 242 
and 243 represent a limited number of individual pixel cells 100 that are included in a first row 
of twelve pixel cells. Similarly, pixel cells 250, 251, 252 and 253 represent a limited number of 
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individual pixel cells 100 that are included in a second row of twelve pixel cells. Using a matrix 
format, an imager array of any size can thus be created by increasing the number of rows and 
columns. Further, Row Select Lines 200, 202, 204 and 206 actually extend to address each of 
the pixel cells in rows 1, 2, 3 and 4 respectively and have been limited in Figure 2 to addressing 
5 the middle set of pixel cells only to provide clarity to the illustration. 

The electrical charges collected in the first row of pixel cells, of which pixel cells 240, 
241, 242 and 243 are illustrative examples, may be serially read out on output line 110 by 
selecting Row Select Line 200 and selecting each individual Column Line 216, 214, 212 and 210 
used to address pixel cells 240, 241, 242 and 243. In one scheme, column lines 216, 214, 212 

10 and 210 are selected in sequential order. At the completion of reading the twelve pixel cells in 
this illustrated first row, the processing continues to serially collect electrical charges in the next 
row (row 2) by selecting Row Select Line 202 and selecting the individual column lines 216, 
214, 212 and 210. The electrical charges collected at the pixel cells of the second row, of which 
pixel cells 250, 251, 252 and 253 are illustrative examples, may then be read out, sequentially for 

15 example, on output line 110. This pixel cell read out process continues, row by row, until the 
last row has been reached. At the conclusion of the pixel cell read out each pixel cell is reset. 
This reset may occur after an individual pixel cell is read, after a row of pixel cells have been 
read or after the entire array imager has been read. The timing of the reset is dependent on the 
specific implementation. 

20 Figure 2 further illustrates an exemplary grouping of individual pixel cells in a P x Q 

array 215, in accordance with the principle of the invention. In this illustrative embodiment 
array 215 is composed of sixteen pixel cells 100 arranged as a 4 x 4 array. In this configuration 
the array 215 is deemed a single entity with a single output 230, which is formed as a composite 
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of the outputs of the pixel cells within the array (in this case, sixteen pixel cells). Array 215 can 
be composed of any number of pixel cells 100 that may be addressed as a single entity. 

In accordance with the principle of the invention, output of array 215 is selected by 
activating Group Select line 220, which causes an output to be generated, concurrently, from 
5 each pixel cell in the array. The concurrently generated outputs are then merged together to form 
a single composite output 230. Similar groups of P x Q arrays having similar merged outputs 
may be created throughout the entire imager array. These Px Q arrays may then be 
superimposed upon the individual addressing lines, represented by Row Select Lines 200, 202, 
204, and 206 and Column Select Lines 216, 214, 212 and 210. In this overlapping of addressing 

10 lines, outputs of the individual pixel cells, 240-243, 250-253, 260-263 and 270-273 and the 
composite output 230 may thus be obtained. These outputs may be obtained substantially 
concurrently or sequentially. 

Figure 3a illustrates one exemplary embodiment of logic circuitry used to achieve pixel 
cell dual read out in accordance with the principles of the invention. As illustrated a dual output 

15 pixel 101 has a first output that is connected to corresponding amplifiers 106 and row and 

column switches 104 and 102, respectively, and second output that is grouped together to form a 
single composite output through amplifier 320. 

As illustrated in this exemplary embodiment, six dual output pixel cells 101 are arranged" 
in a 2 x 3 array wherein the two rows are labeled row m and row m+1, respectively, and the three 

20 columns are labeled n, n+1 and n+2, respectively. It should be understood that it is not intended 
to limit the array size to that illustrated. The number of pixel cells 101 in each row and in each 
column may be expanded to include any number of pixel cells. Hence PxQ arrays of any 
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rectangular or square arrangement can be created in accordance with the principle of the 
invention. 

In this illustrated configuration, a pixel cell output may be transmitted to output line 110 
by activating, or closing the appropriate row and column switches. The output of pixel cell 101a 
5 is directed to output line 110 by selecting Row Select line 200 and Column Select line 216. The 
selection of Row Select Line 200 causes switches 104a through 104c to close, and the selection 
of Column Select line 216 causes switch 102a to close. Similarly, the output of pixel cell 101c is 
directed to Output line 110 by selecting Row Select line 200 and Column Select line 210. The 
selection of Column Select line 210 causes switch 102c to close. In this manner of addressing, 

10 each pixel cell within Row n may be read out on output line 110 by selecting line 210 and by 
selecting, in a sequential order, each Column Select line in the dual resolution imager (Column 
Select lines 216, 214 and 210). 

Further, in accordance with the principle of the invention, the composite output of array 
215 can be output to Group output 230 by selecting Group Row Select 300 and Group Column 

15 Select 360. The selection of Group Row Select 300 and Group Column Select 360 may be 
performed concurrently, or sequentially, with the selection of Row Select lines and Column 
Select lines. In either concurrent or sequential mode, selecting Group Row Select 300 causes 
switches 310a through 310f to close, and the second outputs of the corresponding pixel cells 
(101a through lOlf) are combined in amplifier 320. When Group Column Select 360 is selected, 

20 switch 330 closes and the combined output is made available as Group Output line 400 through 
amplifier 340. 

In the illustrated embodiment, amplifier 320 is a summing amplifier that sums the values 
on the corresponding input lines. In another embodiment, the output of amplifier 320 may be an 
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average of the values on the corresponding input lines. In still another embodiment, as 
illustrated in Figure 3b, each second output of a dual output pixel cell can be directed to an 
amplifier that adjusts the pixel cell output value by a pre-determined amount. Illustrated in 
Figure 3b are amplifiers 380a through 380f that adjust a corresponding pixel cell second output. 
5 When the gains of amplifiers 380a through 380f are set substantially equal, the pixel cell outputs 
provided to summing amplifier 320 correspond to the level of energy collected in the cells. In 
another embodiment, each pixel cell second output may be adjusted based on an amplifier gain 
value that is determined from the position of the pixel cell within the P x Q array. In this 
embodiment, pixel cells closer to the center of the P x Q array have a greater weight than pixel 

10 cells toward the edges of the array. For example, the amplifier gains may be Gaussian 
distributed based on the position of the pixel cell in the array. 

Figure 4 illustrates one exemplary embodiment of an imager array architecture in 
accordance with the principle of the invention. Illustrated is a 6 x 4-pixel imager array that is 
divided into four pixel cell groups using a 3 x 2 pixel cell group, similar to pixel cell group 215 

15 of Figure 3a. As should be understood, the illustrated array is merely representative of an imager 
in accordance with the principles of the invention and that array imagers may be constructed of 
any number of rows and columns of pixel cells. 

In the illustrated embodiment, individual dual output pixel cells 101a through lOlf 
constitute a first pixel group that is addressable by selecting Group Row Select line 300 and 

20 Group Column Select line 360. The selection of Group Row Select 300 and Group Column 

Select 360 cause switches 301 and 330 to close and output 230 is thus made available on Group 
Output 400. Similarly, pixel cells lOlg through 1011 constitute a second pixel group that is 
addressable by selecting Group Row Select line 300 and Group Column Select line 361. The 
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selection of Group Row Select line 300 and Group Column Select line 361 cause switches 301 
and 331, respectively, to close and output 233 is made available on Group Output 400. The two 
remaining pixel groups, similarly, are addressed by Group Row Select 221 and Group Column 
Select 360 and 361, respectively. These two groups generate outputs 232 and 231, respectively, 
5 which are made available on Group Output 400. Thus, by reading the plurality of pixel cell 
groups contained within the pixel array an image may be created that is of lower resolution than 
the capability of the imager array. 

According to the principles of the invention, a dual-output pixel cell is used as a basic 
pixel cell in the imager array. A high-resolution image of a selected region may then be obtained 

10 by reading one output of each pixel cell in the selected region, while a low resolution image may 
be obtained by reading the combined outputs of each pixel cell groups. Thus, in Figure 4, a high 
resolution output can be achieved in a region containing only pixel cells lOla-lOlf and 101m- 
lOlr by reading the outputs of these individual cells while a low-resolution image can be 
concurrently obtained by reading the composite outputs of these same pixel cells. In this 

15 embodiment illustrated, the high-resolution image has a resolution approximately six times better 
than the corresponding low-resolution image. 

Also illustrated in Figure 4 is Group Reset line 410 that may be used to reset each pixel 
cell within at least one pixel group. The single Group Reset line 410 resets all the pixel cells in 
the entire 6 x 4-imager array. However, it should be understood that in another embodiment, a 

20 plurality of Group Reset lines, one for each pixel cell group may also be included. When a 

plurality of Group Reset lines are used, a pixel cell group may be reset after a pixel cell group is 
read out rather than waiting for the entire image to be read. For the same reasons as previously 
discussed, only a single Reset line 108 has been included in the embodiment illustrated in Figure 
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4. It would be understood that means for resetting individual pixel cells, such as 101a, and 101b 
are typically included in imager arrays and the invention may use a single pixel reset switch or a 
plurality of such switches to reset individual pixels after a high resolution output read is 
performed. 

5 Figure 5a represents an exemplary image 500 that is desired for display. As discussed, 

each pixel cell in the imager array captures a portion of the image 500 and each pixel must be 
read out. Using a high resolution output, the time to read every pixel cell may become 
excessively long and the image may appear to move in an irregular manner. Thus, a lower 
resolution image with a shorter Frame time may produce adequate results. Figure 5b illustrates a 

10 digital representation of the image 500 using the low-resolution output. The low -resolution 
output is obtained by processing the composite output of a plurality of pixel cell groups. The 
low-resolution image is advantageous as the Frame time is held to a minimal value. However, 
the image does not display the clarity that is desired. 

Figure 5c illustrates a selected region 530 where a high-resolution image is desired. In 

15 region 530, a high-resolution output is obtained by reading the output of each individual pixel 
cell. The selection of region 530 may be based on object recognition algorithms, user selection 
and intervention, or motion analysis algorithms. For example, a user may use a mouse or 
joystick or keyboard inputs to outline at least one desired area of high-resolution display. In the 
embodiment of a user interface, a joystick is the preferred mode. 

20 Figure 5d illustrates the high-resolution image 540 obtained by reading the first output of 

each pixel cell within region 530. Image 540 utilizes the high-resolution capability of the imager 
array while using only a portion of the time that is required to read the entire image in high- 
resolution mode. In one embodiment Figures 5b and 5d may be displayed as two disjoint images 
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on the same imaging devices. They can also be displayed on different imaging devices. In this 
embodiment, high-resolution details of a selected region of the image may be viewed while the 
low-resolution image 510, as illustrated in Figure 5b, remains available to a user for viewing. 
Accordingly, each pixel cell group in the array is read to obtain a complete low-resolution image. 

5 In another embodiment, the image 540 of Figure 5d may be merged with the image of 

Figure 5b. As illustrated in Figure 5e, the background image 510 remains in low-resolution 
mode while the high-resolution image 540 is concurrently displayed in high-resolution mode. 
Thus, a foveated image 550 is created. This foveated image 550 is advantageous as only those 
regions that are of interest are displayed with high resolution while the Frame time is not 

10 substantially increased over the time to create the entire low-resolution image 510. As the low- 
resolution image 510 and high-resolution image 540 may be created concurrently, the time to 
create the foveated image 550 is not substantially increased over the time to create the low- 
resolution image 510. 

In still another embodiment, when foveated image 550 is desired, those pixel cell groups 
15 within the selected regions of high-resolution imagery need not be read. In this embodiment, the 
scanning logic can prevent the low-resolution pixel groups from being read, as it is known that 
this data will be overwritten. In this mode, the time to read the imager array is reduced as the 
number of pixel groups read is reduced by the size of the selected foveating regions. For 
example, in a 2048 x 2048 CMOS APS imager having arrays composed of 4 x 4 dual output 
20 pixel cells, a high-resolution image would be a full 2048 x 2048 pixels and a low-resolution 

image would be reduced to 512 x 512. The high-resolution image has a frame rate that is sixteen 
times lower than the low-resolution image when equal output clocking frequencies are used. 
Selecting, for example, a high-resolution image area of 400 x 400 pixels, the low-resolution 
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image would be reduced to 412 x 412 ((2048 - 400)/4). Thus, the number of pixel group outputs 
is reduced from 512 to 412. 

Figure 6 illustrates a system incorporating the dual-resolution imager of the present 
invention. An imaging device, in this example, a camera 600, outputs a WFOV image 602 and a 
NFOV image 604 to processor 610. Processor 610 then may output the two images to separate 
displays 620, 630 or combine the images into a composite image that is displayable on a single 
display 640. Processor 610 also provides inputs 608 to imager 600 to define the number and size 
of the NOFV images. 

The examples given herein are presented to enable those skilled in the art to more clearly 
understand and practice the instant invention. The examples should not be considered as 
limitations upon the scope of the invention, but as merely being illustrative and representative of 
the use of the invention. Numerous modifications and alternative embodiments of the invention 
will be apparent to those skilled in the art in view of the foregoing description. Accordingly, this 
description is to be construed as illustrative only and is for the purpose of teaching those skilled 
in the art the best mode of carrying out the invention and is not intended to illustrate all possible 
forms thereof. It is also understood that the words used are words of description, rather than 
limitation, and that details of the structure may be varied substantially without departing from the 
spirit of the invention and the exclusive use of all modifications which come within the scope of 
the appended claims is reserved. 
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WHAT IS CLAIMED 

1. A method for producing a dual resolution readout in an imaging device containing a 
plurality of photo-sensitive electrical devices arranged in a matrix, each photo-sensitive 
electrical device having a first output and a second output comprising the steps of: 

dividing said plurality of photo-sensitive electrical devices into at least one first 
group of photosensitive electrical devices 

generating a corresponding first signal for each of said at least one first group; 
selecting at least one second group of said plurality of photo-sensitive electrical devices, 
each of said photo-sensitive electrical devices within said at least one second group 
generating a corresponding second signal; and 

displaying said first group signals and said second signals 

2. The method as recited in claim 1 wherein said at least one second group is randomly 
positioned with respect to said at least one first group. 

3. The method as recited in claim 2, wherein said at least one second group is designated by 
from the group of object recognition algorithms, user interactive inputs and motion 
analysis. 

4. The method as recited in claim 1 wherein said at least one first group signal is generated 
at a first rate and said corresponding second outputs are scanned at a second rate. 
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5. The method as recited in claim 1 wherein said first signal is generated by the weighted 
accumulation of said photosensitive electrical device first outputs. 

6. An dual resolution imager comprising 

A plurality of photo-sensitive electrical devices each having a first output and a 
second output; 

processing circuits responsive to said first outputs; 
5 processing circuits responsive to said second outputs, wherein said first outputs 

produce a high-resolution image and said second outputs produce a lower resolution 
image. 

7. The imager as recited in claim 6 wherein said second output processing circuit further 
comprises an addressing scheme to concurrently address a plurality of photo-sensitive 
electrical devices. 

8. The imager as recited in claim 7 wherein said second output processing circuit further 
comprises a plurality of amplifiers, one of said amplifiers connected to each of said 
addressable plurality of photo-sensitive electrical devices second outputs. 

9. The imager as recited in claim 6 further comprising a weighting amplifier connected 
between each of said photo-sensitive electrical device second output and said 
corresponding amplifier. 



15 



WO 00/51345 



PCT/US00/04690 



A system to display images using a dual-resolution format comprised of; 

a dual-resolution imager array having a plurality of photosensitive electrical 
devices each having a first output and a second output 

processing circuits for scanning a plurality of first regions at a first rate each of 
said first regions having a plurality of said photosensitive electrical devices, each of said 
first regions generating a first region output; 

processing circuits for selecting at least one second region, 

processing circuits for scanning said at least one second region at a second rate, 
said second region generating a plurality of second region outputs, one said second region 
output for each photo-sensitive electrical device in said at least one second region; 

processing circuits to display said first region outputs and second region outputs. 
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